Driver Emotion and Fatigue State Detection Based on Time Series Fusion

Studies have shown that driver fatigue or unpleasant emotions significantly increase driving risks. Detecting and states providing timely warnings can effectively minimize the incidence of traffic accidents. However, existing models rarely combine emotion detection, there is space to improve accuracy recognition. In this paper, we propose a non-invasive efficient detection method for emotional state, which first time them in state. Firstly, captured video image sequences are preprocessed, Dlib (image open source processing library) used locate face regions mark key points; secondly, facial features extracted, indicators, such as eye closure (PERCLOS) yawn frequency calculated using dual-threshold fused by mathematical methods; thirdly, an improved lightweight RM-Xception convolutional neural network introduced identify driver’s state; finally, two indicators based on series obtain comprehensive score evaluating The results show algorithm proposed paper has high accuracy, recognition reaches rate 73.32% Fer2013 dataset. composite fusion comprehensively accurately reflect state different environments make contribution future research field assisted safe driving.